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(57) Abstract: A color image of relatively large size is inserted in audio signal data as a watermark. And yet the watermark embed- 
ded audio signal data can be played back with sound quality very close to that of original audio signal data. The original audio signal 
data are reshaped into a matrix form to correspond to that of a color image file before wavelet transformed while the color image 
file undergoes a format conversion before discrete cosine transformed. The coefficients of the transformed audio signal data and 
the reformatted color image are combined before inverse-matrix-reshaped and subsequently inverse-wavelet-transformed to finally 
generate an audio signal wherein a color-image watermark has been inserted. 
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DIGITAL WATERMARKING METHOD AND APPARATUS 
USING A COLOR IMAGE AS A WATERMARK 



5 TECHNICAL FIELD 

The present invention relates to digital watermarking 
of data, including audio, video and multimedia data. 
Specifically, the invention relates to embedding a watermark 
10 signal of relatively large data size, such as a color image, 
into digital audio data. 



BACKGROUND ART 

15 

The proliferation of digitized media such as image, 
video and multimedia is creating a need for security systems 
that facilitate the identification of the source of the 
material. Particularly, the internet is increasingly used 

20 for transmitting recorded music in a digitized format. 
Content providers, i.e., owners of such recorded music in 
digital form, have a need to embed into multimedia data a 
predetermined mark, which can subsequently be detected by 
software and/or hardware devices for purposes of 

25 authenticating copyright ownership, control and management 
of the multimedia data. A digital watermarking method has 
been developed as a technique for embedding identifiable 
data into multimedia data. 

Conventionally, watermark signals used for 

30 watermarking audio signal have been relatively simple 
signals such as a sequence of code symbols because, unlike 
inserting a watermark into image or video, inserting a large 
watermark signal would affect the perceptibility of an 
original audio. Moreover, the prior art watermarking 

35 techniques are susceptible to unauthorized removal of 
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watermarks, thereby making hard to trace the origin of a 
copyright protected material. 

5 DISCLOSURE OF THE INVENTION 

It is, therefore, a primary object of the present 
invention to imbed relatively large color image data in 
audio signal data without losing the quality of the audio 

10 signal when it is played back . The watermarking embedding 
method thus provides very high correlation between an 
original audio signal and its watermarked version . This 
object is achieved in part by reshaping audio data structure 
in a matrix to correspond to an image file to be inserted 

15 and performing frequency domain transformations, 

In accordance with one aspect of the present invention, 
there is provided a method for inserting color image 
watermark data into audio data which comprises steps of: 
converting said color image watermark data from a first mode 

20 to a second mode; reshaping said audio data into a matrix to 
correspond to that of said color image watermark data in 
converted in said second mode; wavelet transforming said 
reshaped audio data in a frequency domain to generate first 
spectral coefficients; discrete cosine transforming said 

25 color image data of said second mode in a frequency domain 
to generate second spectral coefficients; combining said 
first and second spectral coefficients; and transforming 
said combined coefficients to generate color image embedded 
audio data. 

30 In accordance with another aspect of the present 

invention, there is provided a digital watermarking 
apparatus for inserting color image watermark data into 
audio data which comprises : a means for converting said 
color image watermark data from a first mode to a second 

35 mode; a means for reshaping said audio data into a matrix to 
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correspond to that of said color image watermark data in 
converted in said second mode; a means for wavelet 
transforming said reshaped audio data in a frequency domain 
to generate first coefficients; a means for discrete cosine 
5 transforming said color image data of said second mode in a 
frequency domain to generate second coefficients; a means 
for combining said first and second coefficients; and a 
means for transforming said combined coefficients to 
generate color image embedded audio data. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 

The above and other objects and features of the 
15 present invention will become apparent from the following 
description of preferred embodiments given in conj unction 
with the accompanying drawings, in which: 

Fig. 1 is a block diagram for inserting a watermark 
signal into audio signal data according to the present 
20 invention; 

Fig. 2 illustrates a Fast Wavelet Transform (FWT); 

Fig. 3 illustrates an inverse Fast Wavelet Transform 
(1FWT) ; 

Fig. 4 shows a color image watermark used in an 
25 experiment; 

Fig. 5 shows the spectrum of an original digital audio 

data; 

Fig. 6 shows the spectrum of a watermark embedded 
digital audio data according to the present invention; 
30 Fig. 7 shows the color image watermark after it is 

extracted from the watermark embedded digital audio data; 
and 

Fig. 8 illustrates a distribution mechanism for 
watermarked audio data according to the present invention. 

35 
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MQDE<OF CARRYING OUT THE INVENTION 

Referring to Fig. 1, the present invention will be 
described .in detail hereinafter- A matrix structure 
5 reshaping unit 10 is configured to reshape original audio 
data in a matrix format identical to that of a color image 
as a watermark to be inserted later on. The original audio 
data in a stream is broken into sections such that the size 
of each section becomes identical to that of an image file . 

10 The image file can be inserted into every section of the 
audio data although intermittent insertion is preferred. 
Meanwhile , a color image format converter 30 converts an 
original color watermark image file of a format such as 
BMP (bitmap) or JPEG (joint photographic coding expert group) 

15 format to a YIQ format, where Y represents luminescence and 
I and Q represent in-phase and quadrature components, 
respectively. The conversion makes possible a fast 

processing of a large color image in a later stage. In the 
present invention, only the Y component will be used. After 

20 this conversion, the scale, size, color information and bit 
structure of the converted image file are extracted by a 
color image information extracting unit 50 and provided to 
the matrix structure reshaping unit 10, which uses this 
information to reshape the original audio data as mentioned 

25 above. 

The reshaped audio data and the YIQ converted image 
data are transformed in the frequency domain by a wavelet 
transform unit 20 and a discrete cosine transform unit 40, 
respectively. A Fourier transform has been used to convert 

30 a watermark in the frequency domain to conceal the content 
of the watermark. In case that an impulse type watermark is 
transformed, its spectral coefficients spread all across the 
spectrum, making it hard to remove it once it is inserted 
into a target data. However, coefficients of the Fourier 

35 transform are complex numbers so that they can not be easily 



WO 01/88883 



PCT/KR00/00645 



added to the target data. Therefore, a discrete cosine 
transformation (DCT) is preferred in the DCT unit 40 because, 
though similar to of the Fourier transform, particularly a 
fast Fourier transform, it results only in real coefficients 
5 in the transformed domain. Conventionally, a DCT has been 
used to encode signals. For example, it is used for 
compression of data in JPEG standard. Basically, it is 
defined as follows: 

10 ^"^I^COS ^ 1 ^ Eq.(l) 

«-o 

where s is the original value of N and t is the converted 
value of AT". The coefficients c are further defined as 
follows : 



15 



20 



25 



30 



c(0) = VT7]V ? c(^)-V27iV for 1< k < N-l Eq. (2) 

Using the two definitions above, a two-dimensional DCT is 
given by the following: 



AM A>-1 



t(hj) = c(hj)2_j 2^K™,n)zos cos \ Eg. (3) 



where N f s, t represent the same parameters as those in one- 
dimensional DCT and c(i f j) is defined as follows: 

c(0,j) = l/n,c(i,0) = l/N,c(i,j) = 2/N for i*0J*0 Eq. (4) 

There is an inverse DCT, which is defined in one-dimension 
and two-dimension as follows, respectively. 

s(n) = ^c(k)t(k)cos \ J Eq. (5) 
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f=0 y=0 ZiV 

To transform the reshaped audio data, a wavelet 
transformation is used in the wavelet transform unit 20. 
5 The Wavelet transform uses wavelets as its basic functions 
just as the Fourier transform uses sine and cosine functions 
as its basic functions. The Wavelet transform falls in two 
categories. One of them is continuous wavelet transform, 
which is defined similarly to the Fourier transform as 
10 follows: 

W(S,T)= [f(t)<p(s 9 t)dt Eq. (7) 

where s is a scaling factor and t is a transform coefficient. 

15 A compressed wavelet has a low scale and expresses 

high frequency components while enlarged wavelet expresses 
low frequency components at a high scale. Thus the scale 
and the frequency are inversely proportional. The number of 
the coefficients generated by continuous wavelet transform 

20 is infinity because they are a function of scaling and 
position displacement. Thus a discrete wavelet transform 
depending on a subset of scaling and position displacement 
is used. More effective algorithm is possible by 

determining scaling and position displacement depending on 

25 dyadic. Nevertheless, still it requires a large volume of 
computations. In order to make up this drawback, a fast 
wavelet transform is suggested. This transform employs a 
conventional two-channel subband coding and a pyramid 
algorithm, and can be applied relatively simply with a so- 

30 called perfectly reconstructive quadrature mirror filter, an 
interconnection among filter banks for inverse transform. 
Fig. 2 and Fig. 3 represent a known fast wavelet transform 
and a known inverse fast wavelet transform, respectively. 

After the transformations as described above, the 
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thus-generated coefficients of the transformed audio data 
and image are each modulated at a matrix coefficient 
modulator 60, for instance, by multiplying a scaling factor, 
in order to minimize the distortion of the original audio as 
5 the result of the watermark: insertion. After the adjustment, 
the two sets of coefficients are added together. Next, the 
combined coefficients are rearranged at the inverse matrix 
structure reshaping unit 70 so that their structure is 
identical to that of the original audio data. Finally, the 

10 rearranged coefficients are inverse wavelet transformed at a 
inverse wavelet transform unit 80 to generate a color image 
embedded audio signal data. 

Fig. 4 represents the spectrum of the original audio 
signal and Fig. 5 shows a color image that is to be inserted 

15 in the audio signal in an experiment. Fig. 6 represents the 
spectrum of the audio signal with the color image embedded 
according to the present invention. As can be easily seen 
from the figures, the two spectrums are almost identical to 
each other with the implication that the original audio 

20 signal was not negatively affected. 1 The correlation between 
the two signals was obtained with the following formula 
widely used in the art: 

Eq. ( 8 ) 

25 

Experiments showed 98.5% correlation, which is well 
above the minimum correlation of 95% because the present 
invention can reduce distortion of the original audio signal 
even when a relatively large image file is inserted. 
30 Furthermore, the extracted watermark has over 75% 
correlation with the original watermark image. Fig. 7 shows 
the watermark of Fig. 5 after it was extracted from the 
watermark embedded signal. The extracted watermark is 
easily identifiable as that of the original watermark. 
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Fig. 8 illustrates one of environments in which the 
present invention is taken advantage of. The watermarked 
audio data, generated from a watermarking unit 100, as 
described above referring to Fig. 1, may be stored in a 
5 database 200 for on-line distribution/downloading to remote 
computers 600 by way of an on-demand server 400. For this, 
such an on-demand server is preferably connected a 
communication network formed by a high-speed communication 
network, for example, such as a B-ISDN or an ATM-LAN . To be 

10 concrete, the communication network is an open network 
represented by the internet 500, or a network using a 
private circuit of personal computer communication or the 
like. Thus access to the server, providing watermarked 
audio data stored in the database, by remote computers or 

15 terminals is possible. The watermark itself can be stored 
in another database 300 for downloading to authorized 
persons in order to verify the extracted watermark is 
genuine . 

The present invention may be further used in the field 

20 of digital image verification and protection, including 
verifying and protecting the integrity of digital images 
stored in memory for authentication and security purposes, 
that has recently gained importance and wide recognition. 

It involves determining whether a digital audio/image 

25 had been modified. In this technique, an 

information/watermark is "stamped" in an original 
audio/multi-media data. This stamping technique is unlikely 
to cause audio/visual artifacts in the original data. 

While there has been described and illustrated methods 

30 and systems for inserting a color image watermark data by 
discrete cosine transforming the watermark signal and 
wavelet transforming an original audio data, it will be 
apparent to those skilled in the art that variations and 
modifications are possible without deviating from the broad 

35 principles and teachings of the present invention which 
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should be limited solely by the scope of the claims appended 
hereto. 
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WHAT IS CLAIMED IS: 



1. A method for inserting color image watermark data into 
5 audio data, comprising steps of: 

(a) converting said color image watermark data from a 
first mode to a second mode; 

(b) reshaping said audio data into a matrix to 
correspond to that of said color image watermark data in 

10 converted in said second mode; 

(c) wavelet transforming said reshaped audio data in a 
frequency domain to generate first spectral coefficients; 

(d) discrete cosine transforming said color image data 
of said second mode in a frequency domain to generate second 

15 spectral coefficients; 

(e) combining said first and second spectral 
coefficients; and 

(f) transforming said combined coefficients to 
generate color image embedded audio data. 

20 

2. The method of claim 1, further comprising steps of: 

(g) obtaining information about said color image 
watermark data; and 

25 (h) using said information in said step (b) . 

3. The method of claim 2 T wherein said information 
includes one or more of the image size, the number of colors 

30 in each pixel and bit array structure. 

4 . The method of claim 1 r wherein said step ( f ) includes 
steps of: 

35 (fl) inverse reshaping said combined coefficients into 
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an original data structure of said audio data; and 

(f2) inverse wavelet transforming said reshaped 
combined coefficients to generate a watermark embedded audio 
data . 

5 

5. The method of claim 1, wherein said first mode is RGB 
mode and said second mode is YIQ mode, where R, G and B 
represent red, green and blue components of a pixel, 

10 respectively, and Y, I and Q represent luminescence, in- 
phase and quadrature components of a pixel, respectively. 

6. A digital watermarking apparatus for inserting color 
15 image watermark data into audio data, comprising: 

a means for converting said color image watermark data 
from a first mode to a second mode; 

a means for reshaping said audio data into a matrix to 
correspond to that of said color image watermark data in 
20 converted in said second mode; 

a means for wavelet transforming said reshaped audio 
data in a frequency domain to generate first coefficients; 

a means for discrete cosine transforming said color 
image data of said second mode in a frequency domain to 
25 generate second coefficients; 

a means for combining said first and second 
coefficients ; and 

a means for transforming said combined coefficients to 
generate color image embedded audio data. 

30 

7. The digital watermarking apparatus of claim 6, further 
comprising : 

a means for obtaining information about said color 
35 image watermark data; and 
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WO 01/88883 



PCT/KR00/00645 



a means for using said information in said means for 
reshaping - 

5 8. The digital watermarking apparatus of claim 7, wherein 
said information includes one or more of the image size , the 
number of colors in each pixel and bit array structure. 

10 9. The digital watermarking apparatus of claim 6, wherein 
said transforming means further includes: 

a means for inverse reshaping said combined 
coefficients into an original data structure of said audio 
data; and 

15 a means for inverse wavelet transforming said reshaped 

combined coefficients to generate watermark embedded audio 
data . 

20 10. The digital watermarking apparatus of claim 6, wherein 
said first mode is RGB mode and said second mode is YIQ mode, 
where R, G and B represent red, green and blue components of 
a pixel, respectively, and Y, I and Q represent luminescence, 
in-phase and quadrature components of a pixel, respectively. 

25 

11. The digital watermarking apparatus of claim 6, further 
comprising a storage means for storing said watermark 
embedded audio data and said color image watermark. 

30 

12 . The digital watermarking apparatus of claim 11, 
further comprising a server connected to a communication 
network for communicating said watermark embedded audio data 

35 stored in said storage means to remote terminals coupled to 
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said communication network. 
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